Adding git workflows with test matrix and common workflows#363
Adding git workflows with test matrix and common workflows#363chethanuk wants to merge 3 commits intoawslabs:masterfrom
Conversation
|
Hi, Thanks for introducing these changes! We will have a look within the next two weeks. |
Sure, I will update it. There are few changes, will push it in weekend [when I get some time] |
|
@twollnik Also, When will be Spark 3.1.1 supported by deequ? |
|
see #366 |
twollnik
left a comment
twollnik
left a comment
There was a problem hiding this comment.
This looks super useful overall, thanks! We would like to add just the testing workflow for now to keep the scope of the PR small (and to be honest because we are no github experts and don't feel comfortable maintaining a lot of github config code). Thanks!
.github/workflows/build_test.yml
Outdated
| hadoop: [2.7, 3.2] | ||
| java: [11] | ||
| spark-version: [2.4.7, 3.0.2] # 3.1.1 - not supported yet | ||
| python-version: [3.7, 3.8] |
There was a problem hiding this comment.
Could you please clarify why we need different python version and why do we need python at all?
There was a problem hiding this comment.
Internally I had mono repo both deequ and python-deequ, I will remove it
There was a problem hiding this comment.
On Side note, Having one mono repo for both is better
|
Thanks so much for your quick reaction on this and for making changes. It will likely take some time for us to react to the updated PR, but we will get around to it! |
|
@twollnik Any update? and Also waiting for Spark 3.1 support as well. |
|
Ping!. |
|
Thanks for your patience on this. @sscdotopen could you please take a look and let us know what you think about the proposed changes? |
3 participants
- java: 8 hadoop: 2.7 spark-version: 2.4.7 - java: 8 hadoop: 3.2 spark-version: 3.0.2 - java: 11 hadoop: 3.2 spark-version: 3.0.2 - java: 8 hadoop: 2.7 spark-version: 3.0.2 - java: 11 hadoop: 2.7 spark-version: 3.0.2 - java: 8 hadoop: 3.2 spark-version: 3.1.1 - java: 11 hadoop: 3.2 spark-version: 3.1.1 - java: 8 hadoop: 2.7 spark-version: 3.1.1 - java: 11 hadoop: 2.7 spark-version: 3.1.1cc: @twollnik @TammoR